Progressive Hard Negative Masking: From Global Uniformity to Local Tolerance

Unsupervised contrastive learning has recently become increasingly popular due to its amazing performance without the need for costly annotations. However, indiscriminate sampling of negative pairs is accompanied by uniformity-tolerance dilemma, which especially serious in node-level graph smoothing property convolutional operators. Previous mining strategies that either overly emphasize hard negatives or rely on precise distribution estimation can make minor improvements even degrade such a case. In this paper, we investigate role dilemma and propose novel objective with progressive masking scheme. The proposed objective, as an asymptotically-tightened lower bound mutual information, theoretically empirically demonstrated be capable allowing higher local tolerance stronger effects, thus leading higher-quality embedding distributions considerable improvement downstream node classification tasks.